ADEPT: A DEbiasing PrompT Framework

Several works have proven that finetuning is an applicable approach for debiasing contextualized word embeddings. Similarly, discrete prompts with semantic meanings shown to be effective in tasks. With unfixed mathematical representation at the token level, continuous usually surpass ones providing a pre-trained language model (PLM) additional task-specific information. Despite this, relatively few efforts been made debias PLMs by prompt tuning compared its counterpart. Furthermore, most methods alter PLM's original parameters, major problem need not only decrease bias PLM but also ensure does lose ability. Finetuning typically hard time maintaining this balance, as they tend violently remove of attribute words (like developing our concepts "male" and "female" gender), which leads unstable unpredictable training process. In paper, we propose ADEPT, method using while delicate balance between removing biases ensuring To achieve new criterion inspired manifold learning equip it explicit term optimize tuning. addition, conduct several experiments regard reliability, quality, quantity previously proposed corpus order obtain clearer prototype certain attribute, indicates attribute's position relative distances other on manifold. We evaluate ADEPT widely acknowledged benchmarks downstream tasks, find achieves competitive results (and some cases even improving) further visualize words' correlation before after PLM, give possible explanations visible effects.